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(54) Abstract Title 

Method and system of sound processing 



(57) A device 410 such as a mobile phone utilises a directional microphone array to discriminate voice signals 
from background noises. In order to do this, the location of a sound source relative to the array 412 must be 
known 402. The location of the source may be determined from signals produced in the microphone array but 
if the array is moved during a period of silence, it may take a long time for the source location to be 
reestablished. In order to mitigate this, a platform motion sensor 400 which may include a gyroscope and/or 
an accelerometer is attached to the microphone array. The location of the sound source may be estimated in a 
source location estimator 406 on the basis of the sound signal from the microphone array, and signals from 
the motion sensor. Results of the estimation are used by a sound enhancing processor for enhancing signal 
quality. The enhanced sound and source location estimate may be passed to a speech recognition module. 
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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 

This print takes account of replacement documents submitted after the date of filing to enable the application to comply 
with the formal requirements of the Patents Rules 1995 
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METHOD AND SYSTEM OF SOUND PROCESSING 

Technical Field 

The present invention relates to sound signal processing. In particular, 
the invention relates to enhancing sound signal quality for use in a sound 
processing device. 



Background 

In known methods of enhancing sound signal quality, pick-up of noise 
and reverberation may be reduced by employing an array of remote 
15 microphones. The remote microphones' outputs are processed and then 
combined into a single signal that has an improved signal quality. 



The best processing parameters to use in the microphone array depend 
on the location of the desired sound source relative to the microphones. Many 

20 arrays reject noise by having much higher sensitivity to sound coming from the 
direction (or region) of the desired source than to sounds from elsewhere, 
where noise sources are located. However, the source's relative location is 
often unknown initially, and may change over time. For example, people often 
move their heads whilst speaking, and if an array is in a hand-held device, it is 

25 quite likely to move somewhat during use. 

The location of a sound source relative to a microphone array, according 
to methods known in the art, can be estimated by measuring certain 
characteristics of the signals received from that source by the array of 
30 microphones. An example of one such characteristic is cross-power spectral 
phase. 

The other method of location of a sound source relative to a microphone 
array is estimation by visually imaging the source, with a camera mounted on 
35 the same base as the array. The camera image is processed to locate the 

source within it, using known visual characteristics of the source. For example, a 
speaker's mouth may be located by looking for his/her face in the image. 



Summary of the Invention 



It is an object of the present invention to provide novel methods and a 
5 system for enhancing sound signal quality which overcome the disadvantages 
of the prior art. The methods and system of the invention are for use in portable 
sound processing devices 

The methods and system according to the'present invention allow 
10 processing of sound signals that result in increased quality. The output signal 
has decreased content of acoustic noise and reverberation caused by 
surroundings. The output signal is also less sensitive to changes of location of a 
sound source relative to the microphones. These changes of location are 
caused both by movement of the sound source, and movement of the device 
15 equiped with these microphones. In a typical example, the sound source would 
be a person speaking and the device might be held in his/her hand. 

In accordance with the present invention, there is thus provided a method 
for enhancing sound signal quality. The method comprises steps of receiving 

20 sound from sound source by a microphone array and detecting a movement of 
this microphone array. The movement is detected by a platform motion sensor. 
Current location of the sound source relative to the microphone array is 
estimated on the basis of the sound signal from the microphone array and 
signals from the platform motion sensor. Results of this estimation are used for 

25 enhancing output sound signal quality. 

An advantage of the present invention is that the motion sensor can 
continously track the motion of the microphone array, and use this to adjust the 
current estimate of the relative location of the desired sound source. The 
30 changes of relative location of the sound source and microphone array can be 
tracked even when there is no sound signal. In contrast, the sound-based 
source location estimation can only track changes in source location when 
sound is present. 

35 There are frequent silences during normal speech, not only between 

words or sentences, but also within words. If the microphone array was moved 
during such a silence, and speech then restarted, it would take a relatively long 
time for a sound-based source location estimator on its own to obtain an 
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accurate estimate of the changed location. The result would be an initial 
reduction in output sound quality. The present invention, on the other hand, 
would use information from the motion sensor to continually adjust the source 
location estimate. Hence, when speech restarts after a silence, there is no initial 
5 reduction in output sound quality. This assumes that only the microphone array 
moves significantly, not the speech source. 

In accordance with another aspect of the present invention, there is 
provided a system for enhancing sound signal quality. The system comprises a 
10 microphone array and a platform motion sensor that are connected to a source 
location estimator. Additionally the microphone array is connected to a sound 
enhancing processor. The sound enhancing processor is connected to a source 
location estimator. 

15 

Brief description of the drawings 

The present invention will be understood and appreciated more fully from 
the following detailed description taken in conjunction with the drawings in 
20 which: 

Figure 1 is a flowchart illustrating a method of preparing input signals for 
use in portable sound processing devices in accordance with an embodiment of 
the present invention; 

25 Figure 2 is a flowchart illustrating a method of enhancing quality of output 

sound signals for use in portable sound processing devices in accordance with 
another embodiment of the present invention; 

Figure 3 is a flowchart illustrating a further method of enhancing quality of 
30 output sound signals for use in portable sound processing devices in 
accordance with another embodiment of the present invention; 
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Figure 4 is a schematic illustration of a system for enhancing quality of 
output sound signals operative in accordance with the present invention. 
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Detailed description of the preferred embodiment 



The term a "movement signal" herein below refers to a signal generated 
and provided by a platform motion sensor. 

5 

A microphone array mentioned herein below refers to a set of 
microphones that are a part of a portable device. Such a portable device is 
typically in a user's hand, during use, and may be subject to various rotational 
and translational movements. 

10 

Referring to figures 1 and 4, in step 100 a sound from a sound source 
412, which may be moving, is received by a microphone array 402. In step 102, 
a platform motion sensor 400 detects a movement of the microphone array 402. 
Afterwards, a sound signal is provided to a sound enhancing processor 408 and 
15 to a source location estimator 406. At the same time the movement signal is 
provided to the source location estimator 406. 



In step 1 12 the sound enhancing processor 408 enhances the quality of 
the sound signal. This enhancement may be performed according to methods 
20 known in the art, and is not described further here. 



In step 104, on the basis of this sound signal, an estimation of a location 
of the sound source 412 relative to the microphone array 402 is performed. For 
the purpose of this estimation, long time intervals of the sound signal are 
25 analysed. Simultaneously in step 106, on the basis of the movement signal, an 
estimation of a change of position of the microphone array 402 is performed. 
For the purpose of this estimation, short time intervals of the movement signal 
are analysed. 

30 In step 108, on the basis of results of estimations performed in steps 104 

and 106, a current change of source's relative location is estimated. 

Finally in step 110, the estimated current location of the sound source 
412 relative to the microphone array 402 is obtained, by combining the current 
35 change of source's relative location from step 1 08 with the location of the sound 
source 412 relative to the microphone array 402. 



Results of the estimation obtained in step 110 and the enhanced signal 
received in step 112 can be used for further processing in other devices, e.g. 
speech recognition devices. 

5 In Fig. 2, in step 200, a sound from a sound source 412, which may be 

moving, is received by a microphone array 402. In step 202, platform motion 
sensor 400 detects a movement of the microphone array 402. Afterwards, a 
sound signal is provided to a sound enhancing processor 408 and to a source 
location estimator 406. At the same time the movement signal is provided to the 
1 0 source location estimator 406. 

In step 204, on the basis of this sound signal, an estimation of a location 
of the sound source 412 relative to the microphone array 402 is performed. For 
the purpose of this estimation, long time intervals of the sound signal are 
15 analysed. Results of this estimation are provided to the sound enhancing 
processor 408. 

In step 212 the sound enhancing processor 408 enhances the quality of 
the sound signal using the results of the estimation performed in step 204. 

20 

Simultaneously in step 206, on the basis of the movement signal, an 
estimation of a change of position of the microphone array 402 is performed. 
For the purpose of this estimation, short time intervals of the movement signal 
are analysed. 

25 

In step 208, on the basis of results of estimations performed in steps 204 
and 206, a current change of the source's relative location is estimated. 

Finally, in step 210, an estimation of current location of the sound source 
30 41 2 relative to the microphone array 402 is performed, by combining the current 
change of the source's relative location from step 208 with the location of the 
sound source 412 relative to the microphone array 402. 
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Results of the estimation obtained in step 210 and the enhanced signal 
received in step 212 can be used for further processing in other devices, e.g. 
speech recognition devices. 




In fig. 3, in step 300 a sound from a sound source 412, which may be 
moving, is received by a microphone array 402. In step 302, platform motion 
sensor 400 detects a movement of the microphone array 402. Afterwards, a 
sound signal is provided to a sound enhancing processor 408 and to a source 
5 location estimator 406. At the same time, the movement signal is provided to 
the source location estimator 406. 

In step 304, on the basis of this sound signal, an estimation of a location 
of the sound source 412 relative to the microphone array 402 is performed. For 
10 the purpose of this estimation, long time intervals of the sound signal are 

analysed. Simultaneously, in step 306, on the basis of the movement signal an 
estimation of a change of position of the microphone array 402 is performed. 
For the purpose of this estimation, short time intervals of the movement signal 
are analysed. 

15 

In step 308, on the basis of results of estimations performed in steps 304 
and 306, a current change of the source's relative location is estimated. 

Finally, in step 310, by combining the current change of source's relative 
20 location from step 308 with the location of the sound source 412 relative to the 
microphone array 402, the estimated current location of the sound source 412 
relative to the microphone array 402 is obtained. 

In step 312 the sound enhancing processor 408 enhances the quality of 
25 the sound signal using the results of the estimation performed in step 310. 

The sound signal of enhanced quality obtained in step 312 can be used 
for further processing in other devices, e.g. speech recognition devices. 

30 The long time interval referenced in steps 104, 204 and 304 is typically in 

the range from 10 ms up to 1 s. 

The short time interval referenced in steps 106, 206 and 306 is typically 
in the range from 0.1 us up to 10 [is. 
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Reference is now made to Fig. 4, which depicts a system for enhancing 
quality of an output sound signal, operative in accordance with the embodiment 
of the present invention as illustrated in Fig. 3. 

5 A system for enhancing sound signal quality, according to the present 

invention, comprises a microphone array 402, that is connected to a sound 
enhancing processor 408, and to a source location estimator 406. The system 
also comprises a platform motion sensor 400 that is connected to the source 
location estimator 406. 

10 

In accordance with some embodiments of the present invention, the 
source location estimator 406 may be connected to the sound enhancing 
processor 408. 

15 The microphone array 402 consists of two or more microphones 404. An 

array of at least two microphones can provide output signals that can be used to 
estimate both the direction and distance of a source of sound. 

The platform motion sensor 400 comprises at least one gyroscope and/or 
20 accelerometer. The platform motion sensor 400 is fixed to the microphone array 
402, or to any other part of the portable sound processing device 410 that forms 
one unit with the microphone array 402. 

The source location estimator 406 and sound enhancing processor 408 
25 have respectively a first signal output connection and a second signal output 
connection. 

The invention uses the signal from platform motion sensor 400 to provide 
an indication of movements of the microphone array 402. Therefore it is 

30 necessary that sensor 400 and microphone array 402 be linked so that the 
signal from sensor 400 relates to the motion of microphone array 402. Such a 
link may simply include sensor 400 being attached to array 402, or both being 
attached to the same housing. Therefore, although figure 4 shows elements 
400, 402, 406 and 408 all in one unit, elements 406 and 408 may in fact be 

35 located elsewhere. 



A system in accordance with the invention, or the methods of the 
invention may be used in various portable devices. In particular, the invention is 
usable in portable radio communication devices. Therefore the system may be 
used in a mobile telephone or a portable or mobile PMR radio. The invention 
5 also may be used in a personal digital assistant (PDA) or laptop computer, 
linked for example by a radio or infra-red communication link to a cellular 
network. Such a network may be in a building, or a cellular telephone, or 
UMTS/3G network. 

10 The invention may form part of a Distributed Speech Recognition (DSR) 

system. In such a system, some speech processing would be performed 
remotely, i.e. the processing would be carried out at at least two different 
locations. 




Claims 

1. A method of preparing input signals for use in a portable sound processing 
device, the method comprising the steps of: 

5 - receiving (100) a sound from a sound source (412) by a microphone array 
(402); 

- detecting (102) a movement of said microphone array (402) by a platform 
motion sensor (400); 

- providing a sound signal from said microphone array (402) to a sound 
10 enhancing processor (408) and to a source location estimator (406); 

- providing a movement signal from said platform motion sensor (400) to said 
source location estimator (406); 

- said sound enhancing processor (408) enhancing (112) a sound signal 
quality; 

15 - said source location estimator (406) estimating (1 10) a current location of 
said sound source (412) relative to said microphone array (402); 

- providing said enhanced sound signal to a sound processing device; 

- providing to said sound processing device a result of said estimation of 
said current location of said sound source relative to said microphone array. 

20 

2. A method according to claim 1 wherein said estimation of said current 
location of said sound source relative to said microphone array comprises 
the steps of: 

- estimating (104) a location of said sound source relative to said 
25 microphone array on the basis of said sound signal; 

- estimating (106) a change of position of said microphone array on the basis 
of said movement signal; 

- estimating (108) a current change of a source's relative location on the 
basis of a result of said estimation of said location of said sound source 

30 relative to said microphone array and said estimation of said change of 
position of said microphone array; 

- combining (1 10) said current change of said source's relative location with 
said location of said sound source relative to said microphone array. 

35 3. A method according to claim 2 wherein, for the purpose of said estimation of 
said location of said sound source relative to said microphone array, long 
time intervals of said sound signal are analysed. 




4. A method according to claim 3 wherein said long time intervals are in a 
range from 10ms to 1s. 

5 5. A method according to claim 2 wherein, for the purpose of said estimation of 
said change of position of said microphone array, short time intervals of said 
movement signal are analysed. 

6. A method according to claim 5 wherein said short time intervals are in a 
10 range from 0.1 ^s to 1 Ojxs. 

7. A method according to claim 1 wherein said sound processing device is a 
speech recognition module. 

15 8. A method of enhancing a sound signal quality, for use in a portable sound 
processing device, the method comprising the steps of: 

- receiving (200) a sound from a sound source (412) by a microphone array 
(402); 

- detecting (202) a movement of said microphone array (402) by a platform 
20 motion sensor (400); 

- providing a sound signal from said microphone array (402) to a sound 
enhancing processor (408) and to a source location estimator (406); 

- providing a movement signal from said platform motion sensor (400) to said 
source location estimator (406); 

25 - performing an estimation (204) of a location of said sound source (412) 

relative to said microphone array (402) by said source location estimator (406); 

- providing a result of said estimation of a location of said sound source (412) 
relative to said microphone array (402) to said sound enhancing processor 
(408); 

30 - said sound enhancing processor (408) enhancing (212) said sound signal 
quality; 

- performing (204) an estimation of a current location of said sound source (412) 
relative to said microphone array (402) by said source location estimator (406); 

- providing said enhanced sound signal to a sound processing device; 

35 - providing a result of said estimation of said current location of said sound 

source (412) relative to said microphone array (402) to said sound processing 
device. 
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9. A method according to claim 8 wherein said estimation (204) of a location of 
said sound source relative to said microphone array is performed on the 
basis of said sound signal only. 

5 

10. A method according to claim 9 wherein, for the purpose of said estimation of 
a location of said sound source relative to said microphone array, long time 
intervals of said sound signal are analysed. 

10 1 1 .A method according to claim 10 wherein said long time intervals are in a 
range from 10ms to 1s. 

12. A method according to claim 8 wherein said estimation of said current 
location of said sound source relative to said microphone array comprises 
15 the steps of: 

- estimating (206) a change of position of said microphone array on the basis 
of said movement signal from said platform motion sensor; 

- estimating (208) a current change of a source's relative location on the 
basis of a result of said estimation of said location of said sound source 

20 relative to said microphone array and said estimation of said change of 
position of said microphone array; 

- combining (210) said current change of said source's relative location with 
said location of said sound source relative to said microphone array. 

25 13. A method according to claim 12 wherein, for the purpose of said estimation 
of said change of position of said microphone array, short time intervals of 
said movement signal are analysed. 

14. A method according to claim 13 wherein said short time intervals are in a 
30 range from 0.1 ^s to 10^is. 

15. A method according to claim 8 wherein, for the purpose of said enhancing of 
said sound signal quality, said result of said estimation of a location of said 
sound source relative to said microphone array is used. 

35 

16. A method according to claim 8 wherein said sound processing device is a 
speech recognition module. 




17. A method of enhancing a sound signal quality, for use in a portable sound 
processing device, the method comprising the steps of: 

- receiving (300) a sound from a sound source (412) by a microphone array 
5 (402); 

- detecting (302) a movement of said microphone array (402) by a platform 
motion sensor (400); 

- providing a sound signal from said microphone array (402) to a sound 
enhancing processor (408) and to a source location estimator (406); 

10 - providing a movement signal from said platform motion sensor (400) to said 
source location estimator (406); 

- said source location estimator (406) estimating a current location of said 
sound source (412) relative to said microphone array (402); 

- providing a result of said estimation of said current location of said sound 
15 source (412) relative to said microphone array (402) to said sound 

enhancing processor (408); 

- said sound enhancing processor (408) enhancing said sound signal quality; 

- providing said enhanced sound signal to a sound processing device. 

20 18. A method according to claim 17 wherein said estimation of said current 

location of said sound source relative to said microphone array comprises 
the steps of: 

- estimating (304) a location of said sound source relative to said 
microphone array on the basis of said sound signal; 

25 - estimating (306) a change of position of said microphone array on the basis 
of said movement signal; 

- estimating (308) a current change of a source's relative location on the 
basis of a result of said estimation of said location of said sound source 
relative to said microphone array and said estimation of said change of 

30 position of said microphone array; 

- combining (310) said current change of said source's relative location with 
said location of said sound source relative to said microphone array. 

19. A method according to claim 18 wherein, for the purpose of said estimation 
35 (304) of said location of said sound source relative to said microphone array, 
long time intervals of said sound signal are analysed. 
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20. A method according to claim 19 wherein said long time intervals are in a 
range from 10ms to 1s. 

21 .A method according to claim 18 wherein, for the purpose of said estimation 
5 of said change of position of said microphone array, short time intervals of 
said movement signal are analysed. 

22. A method according to claim 21 wherein said short time intervals are in a 
range from 0.1 \is to 10^s. 

10 

23. A method according to claim 17 wherein, for the purpose of said enhancing 
(312) of said sound signal quality, said result of said estimation of said 
current location of said sound source is used. 

15 24. A method according to claim 17 wherein said sound processing device is a 
speech recognition module. 

25. A system for enhancing sound signal quality for use in a portable sound 
processing device (410), said system comprises: 

20 - a sound enhancing processor (408); 

- a source location estimator (406); 

- a microphone array (402), that is connected to said sound enhancing 
processor (408) and to said source location estimator (406); 

- a platform motion sensor(400) that is connected to said source location 
25 estimator(406). 

26. A system according to claim 25 wherein said source location estimator (406) 
is connected to said sound enhancing processor. 

30 27. A system according to claim 25 or 26 wherein said microphone array (402) 
consists of at least two microphones. 

28. A system according to claim 25 wherein said platform motion sensor (400) is 
fixed to said microphone array. 

35 

29. A system according to any of claims 25 - 27 wherein said platform motion 
sensor (400) consists of at least one gyroscope. 
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30. A system according to any of claims 25 - 27 wherein said platform motion 
sensor (400) consists of at least one accelerometer. 

5 31 .A system according to any of claims 25 - 27 wherein said platform motion 

sensor (400) consists of a set of at least one accelerometer and at least one 
gyroscope. 

32. A mobile telephone, a portable or mobile PMR radio, a personal digital 
10 assistant (PDA), or a laptop computer being adapted to operate in 

accordance with the method of any of claims 1-7. 

33. A mobile telephone, a portable or mobile PMR radio, a personal digital 
assistant (PDA), or a laptop computer being adapted to operate in 

15 accordance with the method of any of claims 8-16. 

34. A mobile telephone, a portable or mobile PMR radio, a personal digital 
assistant (PDA), or a laptop computer being adapted to operate in 
accordance with the method of any of claims 1 7 - 24. 

20 

35. A mobile telephone, a portable or mobile PMR radio, a personal digital 
assistant (PDA), or a laptop computer comprising a system according to any 
of claims 25-31. 
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